As a storage device provided in a general host system, for example, a computer system, there exists a magnetic HDD (Hard Disk Drive) or an SSD (Solid-State Drive) including a nonvolatile semiconductor memory. Although the SSD is classified as a storage, it can also be regarded as a memory system with an extended scale and functions.
A memory system includes, for example, an interface, a first memory block, a second memory block, and a controller. The first memory block stores files as data, and the second memory block functions as a buffer memory in data write/read. The first memory block is nonvolatile and has a large capacity but a low access speed as compared to the second memory block. The second memory block is used to compensate for the speed difference between the communication speed of the interface and the write/read speed of the first memory block. For example, the first memory block is a nonvolatile flash memory, and the second memory block is a volatile DRAM or SRAM. Such a conventional storage type memory system has an arrangement for implementing a data write/read function by addressing.
On the other hand, to efficiently extract data such as another text associated with a given text, a specific bit pattern in a binary file, a specific pattern in a moving image file, or a characteristic voice pattern in a voice file stored in the memory system, a data read function by data designation is preferably imparted. For this purpose, a method is used to store not only normal data but also metadata associated with the data together and refer to the metadata to obtain desired data.
Metadata management methods are roughly classified into two types: a database type having a table format and a key-value store (KVS) in which data have a one-to-one correspondence. In the KVS, when a key is given as a search request, a value corresponding to the key is output.
To implement the KVS by the conventional system, management of data stored in the memory system or repetitive data input/output processing of expanding a plurality of metadata on the main storage device (DRAM) of the host system, performing an operation using the CPU (Central Processing Unit), and reading out the data from the storage again and collating them is executed.
The KVS in the conventional system and the problems thereof will be described.
To implement the KVS in a conventional SSD, data is stored as a file, and metadata that is key-value data (or key-value pair) belonging to the data is also stored as a file. That is, the key-value store is implemented by the host system equal to or higher than the level of the file system, that is, a file system or an application installed in the OS.
In this case, the key-value store can be implemented by a general-purpose hardware arrangement. However, since metadata is handled like normal data, the metadata read/write or search operation is performed after the host system has read out the metadata file to the main memory (for example, DRAM). This poses, for example, at least the following three problems.
First, the file access performance deteriorates. Since the size of the main memory is smaller than the size of the SSD in general, a metadata file larger than the main memory size cannot be handled in one go. For this reason, the metadata file is divided on a key basis, for example. A metadata file of an easy-to-handle size is read out to the main memory as needed and used. This process is repeated until a necessary key-value is obtained. File access to the SSD occurs as many times as the number of metadata files. If the file access speed of the SSD is lower than the speed of the metadata read request, the host system and the local system (memory system) are rate-limited as a whole.
Second, the load on the CPU increases. Since all of metadata creation, management, and collation (search) are performed by the CPU, a load is imposed on the CPU during metadata processing. Especially, when data is updated, corresponding metadata needs to be searched from metadata files and updated because metadata is created in correspondence with data. In addition, since a metadata search also needs to be performed by the CPU fully using a software algorithm, a load for metadata management is newly imposed on the CPU.
Third, the load on the bus or interface increases. As the result of the first and second problems, the host system and the local system (memory system) frequently exchange metadata information. Hence, the traffic on the bus and interface increases.